Distribution-Free Learning of Graphical Model Structure in Continuous Domains
نویسنده
چکیده
In this paper we present a probabilistic non-parametric conditional independence test of X and Y given a third variable Z in domains where X, Y , and Z are continuous. This test can be used for the induction of the structure of a graphical model (such as a Bayesian or Markov network) from experimental data. We also provide an effective method for calculating it from data. We show that our method works well in practice on artificial benchmark data sets constructed from a diverse set of functions. We also demonstrate learning of the structure of a graphical model in a continuous domain from real-world data, to our knowledge for the first time using independence-based methods and without any distributional assumptions. 1.1 Motivation and Related Work Conditional independence of X and Y given a third variable Z is defined as independence of X and Y for every value z of Z almost surely i.e., except a subset of zero probability (Lauritzen, 1996). In this paper we address the problem of testing for conditional independence when the variables X, Y , and Z are continuous. Such a test can be used as a key building block in a wide range of algorithms, ranging from from simple variable selection to learning the structure of a graphical model from data. The latter is our main motivation in this paper. Existing tests depend on strong assumptions on the underlying distribution, such as linear models with Gaussian errors (e.g., Spirtes et al. (1993)). There are many real-life cases where these assumptions fail (e.g., stock market prices, biometric variables, weather status, etc). We do not make such distributional assumptions and thus our test is non-parametric. One class of algorithms for learning the structure of graphical models (such as Bayesian and Markov networks (Pearl, 1997)) from data uses the fact that it implies that a set of conditional independence statements hold in the domain it is modeling. They exploit this property by using the outcome of a set of statistical conditional independence tests to make inferences about the structure. Assuming no errors in these tests, the idea is to constrain, if possible, the set of possible structures that satisfy the conditional independencies that are present in the data to a singleton, and infer that structure as the only possible one.1 For this reason these algorithms are called constraint-based or independence-based. Testing for conditional independence in discrete domains is straightforward. Perhaps the most common method is the χ2 (chi-square) test of independence. In continuous domains, without making distributional assumptions the problem is more difficult. The standard approach is to discretize the continuous variables and perform a discrete test. However, this has to be done with care. For example, Fig. 1.1 depicts two very different situations where X and Y are dependent (left) and independent (right) produce two very similar histograms. The multi-resolution test of Margaritis and Thrun (2001), outlined in the next section, has been developed to address cases such as this. In this paper we extend it to a conditional version that carefully discretizes the Z axis, performs an independence test in each This is provably correct under certain important assumptions. These are: (a) No unobserved variables exist in the domain, (b) variables are conditionally independent of their non-descendants given their parents, and (c) parameters do not take certain zerosupport values that introduce accidental independencies not implied by the structure.
منابع مشابه
Image alignment via kernelized feature learning
Machine learning is an application of artificial intelligence that is able to automatically learn and improve from experience without being explicitly programmed. The primary assumption for most of the machine learning algorithms is that the training set (source domain) and the test set (target domain) follow from the same probability distribution. However, in most of the real-world application...
متن کاملDistribution-Free Learning of Bayesian Network Structure in Continuous Domains
In this paper we present a method for learning the structure of Bayesian networks (BNs) without making any assumptions on the probability distribution of the domain. This is mainly useful for continuous domains, where there is little guidance and many choices for the parametric distribution families to be used for the local conditional probabilities of the Bayesian network, and only a few have ...
متن کاملInference-less Density Estimation using Copula Bayesian Networks
We consider learning continuous probabilistic graphical models in the face of missing data. For non-Gaussian models, learning the parameters and structure of such models depends on our ability to perform efficient inference, and can be prohibitive even for relatively modest domains. Recently, we introduced the Copula Bayesian Network (CBN) density model a flexible framework that captures comple...
متن کاملThe von Mises Graphical Model: Structure Learning (CMU-CS-11-108 / CMU-CB-11-100)
The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for structure learning using L1 regularization. We show that the learning algorithm is both consistent and efficient. We also introduce a simple inference algorithm based on Gibbs sampling....
متن کاملThe von Mises Graphical Model: Structure Learning
The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for structure learning using L1 regularization. We show that the learning algorithm is both consistent and efficient. We also introduce a simple inference algorithm based on Gibbs sampling....
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004